The present invention relates to testing of software applications. More specifically, the invention relates to testing subsystems of a platform for software applications.
Software applications are commonly designed to be executed on many different platforms. Each platform can have different operating systems, device drivers, file systems, cluster software and interconnects, storage hardware, software and host bus adaptors (HBAs), and the like. It is an extremely challenging task to test subsystems (or changes thereof) of a platform for a software application.
In order to illustrate why this task is so challenging, FIG. 1 shows a diagram of examples of the capability and tested areas for a platform. Capability space 1 represents all the capabilities of a platform. The capability space is quite large and is not generally tested by platform vendors. However, it is common for platform vendors to test certain areas, which are shown as platform vendor tested areas 3.
Application utility space 5 represents the capabilities of platform 1 that are utilized by a software application. As shown, application utility space 5 overlaps some platform vendor tested areas but is, for the most part, not tested by the platform vendors.
An instance area 7 represents the capabilities that are utilized by the application at a specific moment in time. As shown, this area is smaller than application utility space 5. As the application executes, instant utilization space 7 can shrink, grow and move around within application utilization space 5.
Currently, some software applications are tested to cover a test area 9. As shown, the test area covers what may be utilized by an application at one instance in time. However, test coverage 9 can be substantially less than application utility space 5. Therefore, a running application will often exceed the bounds of test area 9 utilizing untested capabilities so there is a risk of failure and error.
Testing tools exist (e.g., IOZONE) that allow users to do performance testing of subsystems (e.g., file systems). These tools typically produce comparative performance rates between various systems but do not provide debug information if something goes wrong. Moreover, these tools are not designed with an application in mind. Therefore, these tools are similar to the test performed by platform vendors in that they are generic and not designed for testing subsystems on a platform for a specific software application.
It would be beneficial to have innovative techniques for testing software applications subsystems on a platform for software applications. Additionally, it would be beneficial to test the full application utility space (and perhaps more).